Automatic Detection of Perilunate and Lunate Dislocations on Wrist Radiographs Using Deep Learning.

SUMMARY: Delayed or missed diagnosis of perilunate or lunate dislocations can lead to significant morbidity. Advances in computer vision provide an opportunity to improve diagnostic performance. In this study, a deep learning algorithm was utilized for detection of perilunate and lunate dislocations on lateral wrist radiographs. A total of 435 lateral wrist radiographs were labeled as normal or pathologic (perilunate or lunate dislocation). The lunate in each radiograph was segmented with a rectangular bounding box. Images were partitioned into training and test sets. Two neural networks, consisting of an object detector followed by an image classifier, were applied in series. First, the object detection module was used to localize the lunate. Next, the image classifier performed a binary classification for normal or pathologic. The accuracy, sensitivity, and specificity of the overall system were evaluated. A receiver operating characteristic (ROC) curve and the associated area under the curve (AUC) were used to demonstrate the overall performance of the computer vision algorithm. The lunate object detector was 97.0% accurate at identifying the lunate. Accuracy was 98.7% among the sub-group of normal wrist radiographs, and 91.3% among the sub-group of wrist radiographs with perilunate/lunate dislocations. The perilunate/lunate dislocation classifier had a sensitivity (recall) of 93.8%, specificity of 93.3%, and accuracy of 93.4%. The AUC was 0.986. We have developed a proof-of-concept computer vision system for diagnosis of perilunate/lunate dislocations on lateral wrist radiographs. This novel deep learning algorithm has potential to improve clinical sensitivity to ultimately prevent delayed or missed diagnosis of these injuries.

Long-term survival in patients with post-LVAD right ventricular failure: multi-state modelling with competing outcomes of heart transplant.

BACKGROUND: Multicenter data on long term survival following LVAD implantation that make use of contemporary definitions of RV failure are limited. Furthermore, traditional survival analyses censor patients who receive a bridge to heart transplant. Here we compare the outcomes of LVAD patients who develop post-operative RV failure accounting for the transitional probability of receiving an interim heart transplantation. METHODS: We use a retrospective cohort of LVAD patients sourced from multiple high-volume centers based in the United States. Five- and ten-year survival accounting for transition probabilities of receiving a heart transplant were calculated using a multi-state Aalen Johansen survival model. RESULTS: Of the 897 patients included in the study, 238 (26.5%) developed post-operative RV failure at index hospitalization. At 10 years the probability of death with post-op RV failure was 79.28% vs 61.70% in patients without (HR 2.10; 95% CI 1.72 - 2.57; p = < .001). Though not significant, patients with RV failure were less likely to be bridged to a heart transplant (HR 0.87, p = .4). Once transplanted the risk of death between both patient groups remained equivalent; the probability of death after a heart transplant was 3.97% in those with post-operative RV failure shortly after index LVAD implant, as compared to 14.71% in those without. CONCLUSIONS AND RELEVANCE: Long-term durable mechanical circulatory support is associated with significantly higher mortality in patients who develop post-operative RV failure. Improving outcomes may necessitate expeditious bridge to heart transplant wherever appropriate, along with critical reassessment of organ allocation policies.

The Project Baseline Health Study: a step towards a broader mission to map human health.

The Project Baseline Health Study (PBHS) was launched to map human health through a comprehensive understanding of both the health of an individual and how it relates to the broader population. The study will contribute to the creation of a biomedical information system that accounts for the highly complex interplay of biological, behavioral, environmental, and social systems. The PBHS is a prospective, multicenter, longitudinal cohort study that aims to enroll thousands of participants with diverse backgrounds who are representative of the entire health spectrum. Enrolled participants will be evaluated serially using clinical, molecular, imaging, sensor, self-reported, behavioral, psychological, environmental, and other health-related measurements. An initial deeply phenotyped cohort will inform the development of a large, expanded virtual cohort. The PBHS will contribute to precision health and medicine by integrating state of the art testing, longitudinal monitoring and participant engagement, and by contributing to the development of an improved platform for data sharing and analysis.

Impact of a deep learning assistant on the histopathologic classification of liver cancer.

Artificial intelligence (AI) algorithms continue to rival human performance on a variety of clinical tasks, while their actual impact on human diagnosticians, when incorporated into clinical workflows, remains relatively unexplored. In this study, we developed a deep learning-based assistant to help pathologists differentiate between two subtypes of primary liver cancer, hepatocellular carcinoma and cholangiocarcinoma, on hematoxylin and eosin-stained whole-slide images (WSI), and evaluated its effect on the diagnostic performance of 11 pathologists with varying levels of expertise. Our model achieved accuracies of 0.885 on a validation set of 26 WSI, and 0.842 on an independent test set of 80 WSI. Although use of the assistant did not change the mean accuracy of the 11 pathologists (p = 0.184, OR = 1.281), it significantly improved the accuracy (p = 0.045, OR = 1.499) of a subset of nine pathologists who fell within well-defined experience levels (GI subspecialists, non-GI subspecialists, and trainees). In the assisted state, model accuracy significantly impacted the diagnostic decisions of all 11 pathologists. As expected, when the model's prediction was correct, assistance significantly improved accuracy (p = 0.000, OR = 4.289), whereas when the model's prediction was incorrect, assistance significantly decreased accuracy (p = 0.000, OR = 0.253), with both effects holding across all pathologist experience levels and case difficulty levels. Our results highlight the challenges of translating AI models into the clinical setting, and emphasize the importance of taking into account potential unintended negative consequences of model assistance when designing and testing medical AI-assistance tools.

Deep learning for chest radiograph diagnosis: A retrospective comparison of the CheXNeXt algorithm to practicing radiologists.

BACKGROUND: Chest radiograph interpretation is critical for the detection of thoracic diseases, including tuberculosis and lung cancer, which affect millions of people worldwide each year. This time-consuming task typically requires expert radiologists to read the images, leading to fatigue-based diagnostic error and lack of diagnostic expertise in areas of the world where radiologists are not available. Recently, deep learning approaches have been able to achieve expert-level performance in medical image interpretation tasks, powered by large network architectures and fueled by the emergence of large labeled datasets. The purpose of this study is to investigate the performance of a deep learning algorithm on the detection of pathologies in chest radiographs compared with practicing radiologists. METHODS AND FINDINGS: We developed CheXNeXt, a convolutional neural network to concurrently detect the presence of 14 different pathologies, including pneumonia, pleural effusion, pulmonary masses, and nodules in frontal-view chest radiographs. CheXNeXt was trained and internally validated on the ChestX-ray8 dataset, with a held-out validation set consisting of 420 images, sampled to contain at least 50 cases of each of the original pathology labels. On this validation set, the majority vote of a panel of 3 board-certified cardiothoracic specialist radiologists served as reference standard. We compared CheXNeXt's discriminative performance on the validation set to the performance of 9 radiologists using the area under the receiver operating characteristic curve (AUC). The radiologists included 6 board-certified radiologists (average experience 12 years, range 4-28 years) and 3 senior radiology residents, from 3 academic institutions. We found that CheXNeXt achieved radiologist-level performance on 11 pathologies and did not achieve radiologist-level performance on 3 pathologies. The radiologists achieved statistically significantly higher AUC performance on cardiomegaly, emphysema, and hiatal hernia, with AUCs of 0.888 (95% confidence interval [CI] 0.863-0.910), 0.911 (95% CI 0.866-0.947), and 0.985 (95% CI 0.974-0.991), respectively, whereas CheXNeXt's AUCs were 0.831 (95% CI 0.790-0.870), 0.704 (95% CI 0.567-0.833), and 0.851 (95% CI 0.785-0.909), respectively. CheXNeXt performed better than radiologists in detecting atelectasis, with an AUC of 0.862 (95% CI 0.825-0.895), statistically significantly higher than radiologists' AUC of 0.808 (95% CI 0.777-0.838); there were no statistically significant differences in AUCs for the other 10 pathologies. The average time to interpret the 420 images in the validation set was substantially longer for the radiologists (240 minutes) than for CheXNeXt (1.5 minutes). The main limitations of our study are that neither CheXNeXt nor the radiologists were permitted to use patient history or review prior examinations and that evaluation was limited to a dataset from a single institution. CONCLUSIONS: In this study, we developed and validated a deep learning algorithm that classified clinically important abnormalities in chest radiographs at a performance level comparable to practicing radiologists. Once tested prospectively in clinical settings, the algorithm could have the potential to expand patient access to chest radiograph diagnostics.

Bone Tumor Diagnosis Using a Naïve Bayesian Model of Demographic and Radiographic Features.

Because many bone tumors have a variety of appearances and are uncommon, few radiologists develop sufficient expertise to guide optimal management. Bayesian inference can guide decision-making by computing probabilities of multiple diagnoses to generate a differential. We built and validated a naïve Bayes machine (NBM) that processes 18 demographic and radiographic features. We reviewed over 1664 analog radiographic cases of bone tumors and selected 811 cases (66 diagnoses) for annotation using a quantitative imaging platform. Leave-one-out cross validation was performed. Primary accuracy was defined as the correct pathological diagnosis as the top machine prediction. Differential accuracy was defined as whether the correct pathological diagnosis was within the top three predictions. For the 29 most common diagnoses (710 cases), primary accuracy was 44%, and differential accuracy was 60%. For the top 10 most common diagnoses (478 cases), primary accuracy was 62%, and differential accuracy was 80%. The machine returned relevant diagnoses for the majority of unknown test cases and may be a feasible alternative to machine learning approaches such as deep neural networks or support vector machines that typically require larger training data (our model required a minimum of five samples per diagnosis) and are "black boxes" (our model can provide details of probability calculations to identify features that most significantly contribute to truth diagnoses). Finally, our Bayes model was designed to scale and "learn" from external data, enabling incorporation of outside knowledge such as Dahlin's Bone Tumors, a reference of anatomic and demographic statistics of more than 10,000 tumors.

Medicare Imaging Demonstration: Assessing Attributes of Appropriate Use Criteria and Their Influence on Ordering Behavior.

OBJECTIVE: Persistent concern exists about the variable and possibly inappropriate utilization of high-cost imaging tests. The purpose of this study is to assess the influence of appropriate use criteria attributes on altering ambulatory imaging orders deemed inappropriate. MATERIALS AND METHODS: This secondary analysis included Medicare Imaging Demonstration data collected from three health care systems in 2011-2013 via the use of clinical decision support (CDS) during ambulatory imaging order entry. The CDS system captured whether orders were inappropriate per the appropriate use criteria of professional societies and provided advice during the intervention period. For orders deemed inappropriate, we assessed the impact of the availability of alternative test recommendations, conflicts with local best practices, and the strength of evidence for appropriate use criteria on the primary outcome of cancellation or modification of inappropriate orders. Expert review determined conflicts with local best practices for 250 recommendations for abdominal and thoracic CT orders. Strength of evidence was assessed for the 15 most commonly triggered recommendations that were deemed inappropriate. A chi-square test was used for univariate analysis. RESULTS: A total of 1691 of 63,222 imaging test orders (2.7%) were deemed inappropriate during the intervention period; this amount decreased from 364 of 11,675 test orders (3.1%) in the baseline period (p < 0.00001). Of 270 inappropriate recommendations with alternative test recommendations, 28 (10.4%) were modified, compared with four of 1024 inappropriate recommendations without alternatives (0.4%) (p < 0.0001). Seventy-eight of 250 recommendations (31%) conflicted with local best practices, but only six of 69 inappropriate recommendations (9%) conflicted (p < 0.001). No inappropriate recommendations that conflicted with local best practices were modified. All 15 commonly triggered recommendations had an Oxford Centre for Evidence-Based Medicine level of evidence of 5 (i.e., expert opinion). CONCLUSION: Orders for imaging tests that were deemed inappropriate were modified infrequently, more often with alternative recommendations present and only for appropriate use criteria consistent with local best practices.

JOURNAL CLUB: Predictors of Provider Response to Clinical Decision Support: Lessons Learned From the Medicare Imaging Demonstration.

OBJECTIVE: The efficacy of imaging clinical decision support (CDS) varies. Our objective was to identify CDS factors contributing to imaging order cancellation or modification. SUBJECTS AND METHODS: This pre-post study was performed across four institutions participating in the Medicare Imaging Demonstration. The intervention was CDS at order entry for selected outpatient imaging procedures. On the basis of the information entered, computerized alerts indicated to providers whether orders were not covered by guidelines, appropriate, of uncertain appropriateness, or inappropriate according to professional society guidelines. Ordering providers could override or accept CDS. We considered actionable alerts to be those that could generate an immediate order behavior change in the ordering physician (i.e., cancellation of inappropriate orders or modification of orders of uncertain appropriateness that had a recommended alternative). Chi-square and logistic regression identified predictors of order cancellation or modification after an alert. RESULTS: A total of 98,894 radiology orders were entered (83,114 after the intervention). Providers ignored 98.9%, modified 1.1%, and cancelled 0.03% of orders in response to alerts. Actionable alerts had a 10 fold higher rate of modification (8.1% vs 0.7%; p < 0.0001) or cancellation (0.2% vs 0.02%; p < 0.0001) orders compared with nonactionable alerts. Orders from institutions with preexisting imaging CDS had a sevenfold lower rate of cancellation or modification than was seen at sites with newly implemented CDS (1.4% vs 0.2%; p < 0.0001). In multivariate analysis, actionable alerts were 12 times more likely to result in order cancellation or modification. Orders at sites with preexisting CDS were 7.7 times less likely to be cancelled or modified (p < 0.0001). CONCLUSION: Using results from the Medicare Imaging Demonstration project, we identified potential factors that were associated with CDS effect on provider imaging ordering; these findings may have implications for future design of such computerized systems.

Predicting High Imaging Utilization Based on Initial Radiology Reports: A Feasibility Study of Machine Learning.

RATIONALE AND OBJECTIVES: Imaging utilization has significantly increased over the last two decades, and is only recently showing signs of moderating. To help healthcare providers identify patients at risk for high imaging utilization, we developed a prediction model to recognize high imaging utilizers based on their initial imaging reports. MATERIALS AND METHODS: The prediction model uses a machine learning text classification framework. In this study, we used radiology reports from 18,384 patients with at least one abdomen computed tomography study in their imaging record at Stanford Health Care as the training set. We modeled the radiology reports in a vector space and trained a support vector machine classifier for this prediction task. We evaluated our model on a separate test set of 4791 patients. In addition to high prediction accuracy, in our method, we aimed at achieving high specificity to identify patients at high risk for high imaging utilization. RESULTS: Our results (accuracy: 94.0%, sensitivity: 74.4%, specificity: 97.9%, positive predictive value: 87.3%, negative predictive value: 95.1%) show that a prediction model can enable healthcare providers to identify in advance patients who are likely to be high utilizers of imaging services. CONCLUSIONS: Machine learning classifiers developed from narrative radiology reports are feasible methods to predict imaging utilization. Such systems can be used to identify high utilizers, inform future image ordering behavior, and encourage judicious use of imaging.

Ten commandments for effective clinical decision support for imaging: enabling evidence-based practice to improve quality and reduce waste.

OBJECTIVE: We describe best practices for effective imaging clinical decision support (CDS) derived from firsthand experience, extending the Ten Commandments for CDS published a decade ago. Our collective perspective is used to set expectations for providers, health systems, policy makers, payers, and health information technology developers. CONCLUSION: Highlighting unique attributes of effective imaging CDS will help radiologists to successfully lead and optimize the value of the substantial federal and local investments in health information technology in the United States.

Assessment of follow-up completeness and notification preferences for imaging findings of possible cancer: what happens after radiologists submit their reports?

RATIONALE AND OBJECTIVES: To understand the reasons leading to potentially inappropriate management of imaging findings concerning for malignancy and identify optimal methods for communicating these findings to providers. MATERIALS AND METHODS: We identified all abdominal imaging examinations with findings of possible cancer performed on six randomly selected days in August to December 2013. Electronic medical records (EMR) of one patient group were reviewed 3 months after the index examination to determine whether management was appropriate (completed follow-up or documented reason for no follow-up) or potentially inappropriate (no follow-up or no documented reason). Providers of a second patient group were contacted 5-6 days after imaging examinations to determine notification preferences. RESULTS: Among 43 patients in the first group, five (12%) received potentially inappropriate management. Reasons included patient loss to follow-up and provider failure to review imaging results, document known imaging findings, or communicate findings to providers outside the health system. Among 16 providers caring for patients in the second group, 33% were unaware of the findings, 75% preferred to be notified of abnormal findings via e-mail or EMR, 56% wanted an embedded hyperlink enabling immediate follow-up order entry, and only 25% had a system to monitor whether patients had completed ordered testing. CONCLUSIONS: One in eight patients did not receive potentially necessary follow-up care within 3 months of imaging findings of possible cancer. Automated notification of imaging findings and follow-up monitoring not only is desired by providers but can also address many of the reasons we found for inappropriate management.

Clinical decision support for imaging in the era of the Patient Protection and Affordable Care Act.

Imaging clinical decision support (CDS) systems provide evidence for or against imaging procedures ordered within a computerized physician order entry system at the time of the image order. Depending on the pertinent clinical history provided by the ordering clinician, CDS systems can optimize imaging by educating providers on appropriate image order entry and by alerting providers to the results of prior, potentially relevant imaging procedures, thereby reducing redundant imaging. The American Recovery and Reinvestment Act (ARRA) has expedited the adoption of computerized physician order entry and CDS systems in health care through the creation of financial incentives and penalties to promote the "meaningful use" of health IT. Meaningful use represents the latest logical next step in a long chain of legislation promoting the areas of appropriate imaging utilization, accurate reporting, and IT. It is uncertain if large-scale implementation of imaging CDS will lead to improved health care quality, as seen in smaller settings, or to improved patient outcomes. However, imaging CDS enables the correlation of existing imaging evidence with outcome measures, including morbidity, mortality, and short-term imaging-relevant management outcomes (eg, biopsy, chemotherapy). The purposes of this article are to review the legislative sequence relevant to imaging CDS and to give guidance to radiology practices focused on quality and financial performance improvement during this time of accelerating regulatory change.

The diagnostic and economic yield of neuroimaging in neuro-ophthalmology.

BACKGROUND: Diagnostic studies such as computed tomography scans (CT) and magnetic resonance imaging (MRI) are ordered frequently in neuro-ophthalmic practice, although the diagnostic yield and cost-effectiveness of these tests have been studied for only a few conditions. We assessed the diagnostic and economic yield of CT and MRI across all patients evaluated in a neuro-ophthalmology practice. METHODS: This retrospective review included all patients referred by the division of neuro-ophthalmology at the Scheie Eye Institute for CT, CT angiography, MRI, MRA, or magnetic resonance venography over a 12-month period. Abnormal imaging findings were categorized as significant (one that elicited changes in management) and/or relevant (one that related to the patient's neuro-ophthalmic complaint or examination findings). The diagnostic yield of the test ordered was analyzed according to the patient's chief complaint, neuro-ophthalmic examination findings, and indication for imaging. The total costs for each diagnostic group and costs per significant finding were calculated using the global Resource-Based Relative Value Units for each examination from the Centers for Medicare and Medicaid Services Web site. RESULTS: Two hundred eleven imaging studies in 157 patients were evaluated. 28.9% (95% confidence interval, 22.5%-36.2%) of imaging studies had significant abnormalities relevant to the neuro-ophthalmic complaint. Imaging obtained for evaluation of progressive optic nerve dysfunction and cranial nerve palsy had statistically significant higher diagnostic yield than studies performed for other reasons. Total cost of all imaging studies performed was $107,615.72. Cost per clinically significant and relevant finding was $1,764.19. CONCLUSIONS: In comparison to the diagnostic yield of neuroimaging studies in other specialties, CT and MRI of the brain requested by neuro-ophthalmologists provide significant and relevant data at a reasonable cost.